Wearable Computer with Nearby Object Response

ABSTRACT

Exemplary methods and systems relate to detecting physical objects near a substantially transparent head-mounted display (HMD) system and activating a collision-avoidance action to alert a user of the detected objects. Detection techniques may include receiving data from distance and/or relative movement sensors and using this data as a basis for determining an appropriate collision-avoidance action. Exemplary collision-avoidance actions may include de-emphasizing virtual objects displayed on the HMD to provide a less cluttered view of the physical objects through the substantially transparent display and/or presenting new virtual objects.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.14/737,745, filed Jun. 12, 2015, which is a continuation of U.S. patentapplication Ser. No. 13/267,963, filed Oct. 7, 2011, both of which areincorporated by reference herein in their entirety for all purposes.

BACKGROUND

Unless otherwise indicated herein, the materials described in thissection are not prior art to the claims in this application and are notadmitted to be prior art by inclusion in this section.

Computing devices such as personal computers, laptop computers, tabletcomputers, cellular phones, and countless types of Internet-capabledevices are increasingly prevalent in numerous aspects of modern life.Over time, the manner in which these devices are providing informationto users is becoming more intelligent, more efficient, more intuitive,and/or less obtrusive.

The trend toward miniaturization of computing hardware, peripherals, aswell as of sensors, detectors, and image and audio processors, amongother technologies, has helped open up a field sometimes referred to as“wearable computing.” In the area of image and visual processing andproduction, in particular, it has become possible to consider wearabledisplays that place a very small image display element close enough to awearer's (or user's) eye(s) such that the displayed image fills ornearly fills the field of view, and appears as a normal sized image,such as might be displayed on a traditional image display device. Therelevant technology may be referred to as “near-eye displays.”

Near-eye displays are fundamental components of wearable displays, alsosometimes called “head-mounted displays” (HMDs). A head-mounted displayplaces a graphic display or displays close to one or both eyes of awearer. To generate the images on a display, a computer processingsystem may be used. Such displays may occupy a wearer's entire field ofview, or only occupy part of wearer's field of view. Further,head-mounted displays may be as small as a pair of glasses or as largeas a helmet.

Emerging and anticipated uses of wearable displays include applicationsin which users interact in real time with an augmented or virtualreality. Such applications can be mission-critical or safety-critical,such as in a public safety or aviation setting. The applications canalso be recreational, such as interactive gaming.

SUMMARY

In one aspect, an exemplary method involves detecting a physical objectnear a head-mounted display (HMD) system and activating acollision-avoidance action, including modifying the content displayed onthe HMD. To perform these actions, an exemplary method may involveproviding the HMD with a user-interface, which includes a displayed viewregion and a content region that may be not displayed initially. The HMDmay then display the provided user-interface on a substantiallytransparent display so that when an object is detected, contentdisplayed in the view region may be de-emphasized to draw attention toprovide an unobstructed view of the detected object.

In another aspect, an exemplary apparatus may be a wearable computerthat includes a processor and instructions executable by the processor.The instructions may allow the processor to activate acollision-avoidance action in response to determining that a physicalobject represents a potential risk of collision with the wearer of thewearable computer. The risk-determination process may include receivingand processing data from one or more sensors. Additionally, the wearablecomputer may provide a user-interface to an HMD and thecollision-avoidance action may include modifying the provideduser-interface to inform a wearer of the wearable computer of thedetected object.

In another aspect, an exemplary non-transitory computer-readable mediummay contain instructions to activate a collision-avoidance action inresponse to determining that a physical object represents a potentialrisk of collision with an HMD system. The risk-determination process mayinclude receiving and processing data from one or more sensors.Additionally, medium may include instructions to provide auser-interface to the HMD and the collision-avoidance action may includemodifying the provided user-interface.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1A illustrates an example head-mounted display system;

FIG. 1B illustrates an alternate view of the system illustrated in FIG.1A;

FIG. 2A illustrates an example system for receiving, transmitting, anddisplaying data.

FIG. 2B illustrates an example system for receiving, transmitting, anddisplaying data.

FIG. 3 shows a simplified block diagram of an example computer networkinfrastructure.

FIG. 4 shows a simplified block diagram depicting example components ofan example computing system.

FIG. 5A shows aspects of an example user-interface.

FIG. 5B shows aspects of an example user-interface after receivingmovement data corresponding to an upward movement.

FIG. 5C shows aspects of an example user-interface after selection of aselected content object.

FIG. 5D shows aspects of an example user-interface after receiving inputdata corresponding to a user input.

FIG. 6 is a simplified flow chart illustrating a method according to anexemplary embodiment.

FIG. 7A illustrates an exemplary user-interface and a portion of thephysical field of view beyond the display before activatingcollision-avoidance actions.

FIG. 7B illustrates an exemplary user-interface and a portion of thephysical field of view beyond the display after activating an exemplarycollision-avoidance action.

FIG. 7C illustrates an exemplary user-interface and a portion of thephysical field of view beyond the display after activating an exemplarycollision-avoidance action.

FIG. 7D illustrates an exemplary user-interface and a portion of thephysical field of view beyond the display after activating an exemplarycollision-avoidance action.

FIG. 7E illustrates an exemplary user-interface and a portion of thephysical field of view beyond the display after activating an exemplarycollision-avoidance action.

FIG. 7F illustrates an exemplary user-interface and a portion of thephysical field of view beyond the display after activating an exemplarycollision-avoidance action.

FIG. 7G illustrates an exemplary user-interface and a portion of thephysical field of view beyond the display after activating an exemplarycollision-avoidance action.

FIG. 7H illustrates an exemplary user-interface and a portion of thephysical field of view beyond the display after activating an exemplarycollision-avoidance action.

DETAILED DESCRIPTION

In the following detailed description, reference is made to theaccompanying figures, which form a part thereof. In the figures, similarsymbols typically identify similar components, unless context dictatesotherwise. The illustrative embodiments described in the detaileddescription, figures, and claims are not meant to be limiting. Otherembodiments may be utilized, and other changes may be made, withoutdeparting from the spirit or scope of the subject matter presentedherein. It will be readily understood that the aspects of the presentdisclosure, as generally described herein, and illustrated in thefigures, can be arranged, substituted, combined, separated, and designedin a wide variety of different configurations, all of which arecontemplated herein.

Exemplary methods and systems are described herein. It should beunderstood that the word “exemplary” is used herein to mean “serving asan example, instance, or illustration.” Any embodiment or featuredescribed herein as “exemplary” is not necessarily to be construed aspreferred or advantageous over other embodiments or features. Theexemplary embodiments described herein are not meant to be limiting. Itwill be readily understood that certain aspects of the disclosed systemsand methods can be arranged and combined in a wide variety of differentconfigurations, all of which are contemplated herein.

1. Overview

A wearable computer may include a head-mounted display (HMD) thatpresents virtual objects (e.g., graphical media content such as text,images, application windows, or video) on a substantially transparentdisplay screen. Hence, the wearer of the HMD may maintain a view thephysical world while also viewing and interacting with the virtualobjects that are displayed over their physical field of vision. However,this overlay of virtual objects in the HMD may obstruct the wearer'sview of real-world objects, inhibiting the instinctive response to avoidpotential hazards.

To help avoid this danger, the wearable computer may be configured todetect approaching and/or nearby objects and alert the wearer of thepotential risk. For example, real-world objects may be detected using anumber of different sensors, including cameras, acoustic sensors, and/orlaser-based detection systems, among others. Exemplary alert techniquesmay likewise take on many forms, including de-emphasizing virtualobjects, presenting audio alerts, and/or displaying new virtual objectsto highlight the physical object, among others.

The object detection procedure may run continuously as a backgroundprocess on the wearable computer or it may run only when activated bythe wearer. Alternatively, an automated process may initiate thedetection procedure in response to a determined need. For instance, thewearable computer may use an accelerometer to detect when the wearer isin motion and automatically begin the object detection method,deactivating the process when the wearer again comes to rest. As anotherexample, the computer may monitor the quantity and size of the displayedvirtual objects and automatically activate the object detection methodwhen the display is sufficiently cluttered to obstruct the wearer'sview.

2. Example System and Device Architecture

FIG. 1A illustrates an example system 100 for receiving, transmitting,and displaying data. The system 100 is shown in the form of a wearablecomputing device. While FIG. 1A illustrates a head-mounted device 102 asan example of a wearable computing device, other types of wearablecomputing devices could additionally or alternatively be used. Asillustrated in FIG. 1A, the head-mounted device 102 has frame elementsincluding lens-frames 104, 106 and a center frame support 108, lenselements 110, 112, and extending side-arms 114, 116. The center framesupport 108 and the extending side-arms 114, 116 are configured tosecure the head-mounted device 102 to a user's face via a user's noseand ears, respectively.

Each of the frame elements 104, 106, and 108 and the extending side-arms114, 116 may be formed of a solid structure of plastic and/or metal, ormay be formed of a hollow structure of similar material so as to allowwiring and component interconnects to be internally routed through thehead-mounted device 102. Other materials may be possible as well.

One or more of each of the lens elements 110, 112 may be formed of anymaterial that can suitably display a projected image or graphic. Each ofthe lens elements 110, 112 may also be sufficiently transparent to allowa user to see through the lens element. Combining these two features ofthe lens elements may facilitate an augmented reality or heads-updisplay where the projected image or graphic is superimposed over areal-world view as perceived by the user through the lens elements 110,112.

The extending side-arms 114, 116 may each be projections that extendaway from the lens-frames 104, 106, respectively, and may be positionedbehind a user's ears to secure the head-mounted device 102 to the user.The extending side-arms 114, 116 may further secure the head-mounteddevice 102 to the user by extending around a rear portion of the user'shead. Additionally or alternatively, for example, the system 100 mayconnect to or be affixed within a head-mounted helmet structure. Otherpossibilities exist as well.

The system 100 may also include an on-board computing system 118, avideo camera 120, a sensor 122, and a finger-operable touch pad 124. Theon-board computing system 118 is shown to be positioned on the extendingside-arm 114 of the head-mounted device 102; however, the on-boardcomputing system 118 may be provided on other parts of the head-mounteddevice 102 or may be positioned remote from the head-mounted device 102(e.g., the on-board computing system 118 could be connected by wires orwirelessly connected to the head-mounted device 102). The on-boardcomputing system 118 may include a processor and memory, for example.The on-board computing system 118 may be configured to receive andanalyze data from the video camera 120, the sensor 122, and thefinger-operable touch pad 124 (and possibly from other sensory devices,user-interfaces, or both) and generate images for output by the lenselements 110 and 112. The on-board computing system 118 may additionallyinclude a speaker or a microphone for user input (not shown). An examplecomputing system is further described below in connection with FIG. 4.

The video camera 120 is shown positioned on the extending side-arm 114of the head-mounted device 102; however, the video camera 120 may beprovided on other parts of the head-mounted device 102. The video camera120 may be configured to capture images at various resolutions or atdifferent frame rates. Video cameras with a small form-factor, such asthose used in cell phones or webcams, for example, may be incorporatedinto an example embodiment of the system 100.

Further, although FIG. 1A illustrates one video camera 120, more videocameras may be used, and each may be configured to capture the sameview, or to capture different views. For example, the video camera 120may be forward facing to capture at least a portion of the real-worldview perceived by the user. This forward facing image captured by thevideo camera 120 may then be used to generate an augmented reality wherecomputer generated images appear to interact with the real-world viewperceived by the user.

The sensor 122 is shown on the extending side-arm 116 of thehead-mounted device 102; however, the sensor 122 may be positioned onother parts of the head-mounted device 102. The sensor 122 may includeone or more of a gyroscope or an accelerometer, for example. Othersensing devices may be included within, or in addition to, the sensor122 or other sensing functions may be performed by the sensor 122.

The finger-operable touch pad 124 is shown on the extending side-arm 114of the head-mounted device 102. However, the finger-operable touch pad124 may be positioned on other parts of the head-mounted device 102.Also, more than one finger-operable touch pad may be present on thehead-mounted device 102. The finger-operable touch pad 124 may be usedby a user to input commands. The finger-operable touch pad 124 may senseat least one of a position and a movement of a finger via capacitivesensing, resistance sensing, or a surface acoustic wave process, amongother possibilities. The finger-operable touch pad 124 may be capable ofsensing finger movement in a direction parallel or planar to the padsurface, in a direction normal to the pad surface, or both, and may alsobe capable of sensing a level of pressure applied to the pad surface.The finger-operable touch pad 124 may be formed of one or moretranslucent or transparent insulating layers and one or more translucentor transparent conducting layers. Edges of the finger-operable touch pad124 may be formed to have a raised, indented, or roughened surface, soas to provide tactile feedback to a user when the user's finger reachesthe edge, or other area, of the finger-operable touch pad 124. If morethan one finger-operable touch pad is present, each finger-operabletouch pad may be operated independently, and may provide a differentfunction.

FIG. 1B illustrates an alternate view of the system 100 illustrated inFIG. 1A. As shown in FIG. 1B, the lens elements 110, 112 may act asdisplay elements. The head-mounted device 102 may include a firstprojector 128 coupled to an inside surface of the extending side-arm 116and configured to project a display 130 onto an inside surface of thelens element 112. Additionally or alternatively, a second projector 132may be coupled to an inside surface of the extending side-arm 114 andconfigured to project a display 134 onto an inside surface of the lenselement 110.

The lens elements 110, 112 may act as a combiner in a light projectionsystem and may include a coating that reflects the light projected ontothem from the projectors 128, 132. In some embodiments, a reflectivecoating may be omitted (e.g., when the projectors 128, 132 are scanninglaser devices).

In alternative embodiments, other types of display elements may also beused. For example, the lens elements 110, 112 themselves may include: atransparent or semi-transparent matrix display, such as anelectroluminescent display or a liquid crystal display, one or morewaveguides for delivering an image to the user's eyes, or other opticalelements capable of delivering an in focus near-to-eye image to theuser. A corresponding display driver may be disposed within the frameelements 104, 106 for driving such a matrix display. Alternatively oradditionally, a laser or light emitting diode (LED) source and scanningsystem could be used to draw a raster display directly onto the retinaof one or more of the user's eyes. Other possibilities exist as well.

FIG. 2A illustrates an example system 200 for receiving, transmitting,and displaying data. The system 200 is shown in the form of a wearablecomputing device 202. The wearable computing device 202 may includeframe elements and side-arms such as those described with respect toFIGS. 1A and 1B. The wearable computing device 202 may additionallyinclude an on-board computing system 204 and a video camera 206, such asthose described with respect to FIGS. 1A and 1B. The video camera 206 isshown mounted on a frame of the wearable computing device 202; however,the video camera 206 may be mounted at other positions as well.

As shown in FIG. 2A, the wearable computing device 202 may include asingle display 208 which may be coupled to the device. The display 208may be formed on one of the lens elements of the wearable computingdevice 202, such as a lens element described with respect to FIGS. 1Aand 1B, and may be configured to overlay computer-generated graphics inthe user's view of the physical world. The display 208 is shown to beprovided in a center of a lens of the wearable computing device 202;however, the display 208 may be provided in other positions. The display208 is controllable via the computing system 204 that is coupled to thedisplay 208 via an optical waveguide 210.

FIG. 2B illustrates an example system 220 for receiving, transmitting,and displaying data. The system 220 is shown in the form of a wearablecomputing device 222. The wearable computing device 222 may includeside-arms 223, a center frame support 224, and a bridge portion withnosepiece 225. In the example shown in FIG. 2B, the center frame support224 connects the side-arms 223. The wearable computing device 222 doesnot include lens-frames containing lens elements. The wearable computingdevice 222 may additionally include an on-board computing system 226 anda video camera 228, such as those described with respect to FIGS. 1A and1B.

The wearable computing device 222 may include a single lens element 230that may be coupled to one of the side-arms 223 or the center framesupport 224. The lens element 230 may include a display such as thedisplay described with reference to FIGS. 1A and 1B, and may beconfigured to overlay computer-generated graphics upon the user's viewof the physical world. In one example, the single lens element 230 maybe coupled to a side of the extending side-arm 223. The single lenselement 230 may be positioned in front of or proximate to a user's eyewhen the wearable computing device 222 is worn by a user. For example,the single lens element 230 may be positioned below the center framesupport 224, as shown in FIG. 2B.

FIG. 3 shows a simplified block diagram of an example computer networkinfrastructure. In system 300, a device 310 communicates using acommunication link 320 (e.g., a wired or wireless connection) to aremote device 330. The device 310 may be any type of device that canreceive data and display information corresponding to or associated withthe data. For example, the device 310 may be a heads-up display system,such as the head-mounted device 102, 200, or 220 described withreference to FIGS. 1A-2B.

Thus, the device 310 may include a display system 312 comprising aprocessor 314 and a display 316. The display 316 may be, for example, anoptical see-through display, an optical see-around display, or a videosee-through display. The processor 314 may receive data from the remotedevice 330, and configure the data for display on the display 316. Theprocessor 314 may be any type of processor, such as a micro-processor ora digital signal processor, for example.

The device 310 may further include on-board data storage, such as memory318 coupled to the processor 314. The memory 318 may store software thatcan be accessed and executed by the processor 314, for example.

The remote device 330 may be any type of computing device or transmitterincluding a laptop computer, a mobile telephone, or tablet computingdevice, etc., that is configured to transmit data to the device 310. Theremote device 330 and the device 310 may contain hardware to enable thecommunication link 320, such as processors, transmitters, receivers,antennas, etc.

In FIG. 3, the communication link 320 is illustrated as a wirelessconnection; however, wired connections may also be used. For example,the communication link 320 may be a wired serial bus such as a universalserial bus or a parallel bus, among other connections. The communicationlink 320 may also be a wireless connection using, e.g., Bluetooth® radiotechnology, communication protocols described in IEEE 802.11 (includingany IEEE 802.11 revisions), Cellular technology (such as GSM, CDMA,UMTS, EV-DO, WiMAX, or LTE), or Zigbee® technology, among otherpossibilities. Either of such a wired and/or wireless connection may bea proprietary connection as well. The remote device 330 may beaccessible via the Internet and may include a computing clusterassociated with a particular web service (e.g., social-networking, photosharing, address book, etc.).

As described above in connection with FIGS. 1A-2B, an example wearablecomputing device may include, or may otherwise be communicativelycoupled to, a computing system, such as computing system 118 orcomputing system 204. FIG. 4 shows a simplified block diagram depictingexample components of an example computing system 400. One or both ofthe device 310 and the remote device 330 may take the form of computingsystem 400.

Computing system 400 may include at least one processor 402 and systemmemory 404. In an example embodiment, computing system 400 may include asystem bus 406 that communicatively connects processor 402 and systemmemory 404, as well as other components of computing system 400.Depending on the desired configuration, processor 402 can be any type ofprocessor including, but not limited to, a microprocessor (μP), amicrocontroller (μC), a digital signal processor (DSP), or anycombination thereof. Furthermore, system memory 404 can be of any typeof memory now known or later developed including but not limited tovolatile memory (such as RAM), non-volatile memory (such as ROM, flashmemory, etc.) or any combination thereof.

An example computing system 400 may include various other components aswell. For example, computing system 400 includes an A/V processing unit408 for controlling graphical display 410 and speaker 412 (via A/V port414), one or more communication interfaces 416 for connecting to othercomputing devices 418, and a power supply 420. Graphical display 410 maybe arranged to provide a visual depiction of various input regionsprovided by user-interface module 422. For example, user-interfacemodule 422 may be configured to provide a user-interface, such as theexample user-interface described below in connection with FIGS. 5A-D,and graphical display 410 may be configured to provide a visualdepiction of the user-interface. User-interface module 422 may befurther configured to receive data from and transmit data to (or beotherwise compatible with) one or more user-interface devices 428.

Furthermore, computing system 400 may also include one or more datastorage devices 424, which can be removable storage devices,non-removable storage devices, or a combination thereof. Examples ofremovable storage devices and non-removable storage devices includemagnetic disk devices such as flexible disk drives and hard-disk drives(HDD), optical disk drives such as compact disk (CD) drives or digitalversatile disk (DVD) drives, solid state drives (SSD), and/or any otherstorage device now known or later developed. Computer storage media caninclude volatile and nonvolatile, removable and non-removable mediaimplemented in any method or technology for storage of information, suchas computer readable instructions, data structures, program modules, orother data. For example, computer storage media may take the form ofRAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM,digital versatile disks (DVD) or other optical storage, magneticcassettes, magnetic tape, magnetic disk storage or other magneticstorage devices, or any other medium now known or later developed thatcan be used to store the desired information and which can be accessedby computing system 400.

According to an example embodiment, computing system 400 may includeprogram instructions 426 that are stored in system memory 404 (and/orpossibly in another data-storage medium) and executable by processor 402to facilitate the various functions described herein including, but notlimited to, those functions described with respect to [METHOD FIGURES].Although various components of computing system 400 are shown asdistributed components, it should be understood that any of suchcomponents may be physically integrated and/or distributed according tothe desired configuration of the computing system.

3. Example User-Interface

FIGS. 5A-D show aspects of an example user-interface 500. Theuser-interface 500 may be displayed by, for example, a wearablecomputing device as described above for FIGS. 1A-2B.

An example state of the user-interface 500 is shown in FIG. 5A. Theexample state shown in FIG. 5A may correspond to a first position of thewearable computing device. That is, the user-interface 500 may bedisplayed as shown in FIG. 5A when the wearable computing device is inthe first position. In some embodiments, the first position of thewearable computing device may correspond to a position of the wearablecomputing device when a wearer of the wearable computing device islooking in a direction that is generally parallel to the ground (e.g., aposition that does not correspond to the wearer looking up or lookingdown). Other examples are possible as well.

As shown, the user-interface 500 includes a view region 502. An exampleboundary of the view region 502 is shown by a dotted frame. While theview region 502 is shown to have a landscape shape (in which the viewregion 502 is wider than it is tall), in other embodiments the viewregion 502 may have a portrait or square shape, or may have anon-rectangular shape, such as a circular or elliptical shape. The viewregion 502 may have other shapes as well.

The view region 502 may be, for example, the viewable area between (orencompassing) the upper, lower, left, and right boundaries of a displayon the wearable computing device, or in other words, may substantiallyfill the display in an HMD. As shown, when the wearable computing deviceis in the first position, the view region 502 is substantially empty(e.g., completely empty) of user-interface elements, such that theuser's view of their real-world environment is generally uncluttered,and objects in the user's environment are not obscured.

In some embodiments, the view region 502 may correspond to a field ofview of a wearer of the wearable computing device, and an area outsidethe view region 502 may correspond to an area outside the field of viewof the wearer. In other embodiments, the view region 502 may correspondto a non-peripheral portion of a field of view of a wearer of thewearable computing device, and an area outside the view region 502 maycorrespond to a peripheral portion of the field of view of the wearer.In still other embodiments, the user-interface 500 may be larger than orsubstantially the same as a field of view of a wearer of the wearablecomputing device, and the field of view of the wearer may be larger thanor substantially the same size as the view region 502. The view region502 may take other forms as well.

Accordingly, the portions of the user-interface 500 outside of the viewregion 502 may be outside of or in a peripheral portion of a field ofview of a wearer of the wearable computing device. For example, asshown, a menu 504 may be outside of or in a peripheral portion of thefield of view of the user in the user-interface 500. While the menu 504is shown to be not visible in the view region 502, in some embodimentsthe menu 504 may be partially visible in the view region 502.

In some embodiments, the wearable computing device may be configured toreceive movement data corresponding to, for example, an upward movementof the wearable computing device to a position above the first position.In these embodiments, the wearable computing device may, in response toreceiving the movement data corresponding to the upward movement, causeone or both of the view region 502 and the menu 504 to move such thatthe menu 504 becomes more visible in the view region 502. For example,the wearable computing device may cause the view region 502 to moveupward and may cause the menu 504 to move downward. The view region 502and the menu 504 may move the same amount, or may move differentamounts. In one embodiment, the menu 504 may move further than the viewregion 502. As another example, the wearable computing device may causeonly the menu 504 to move. Other examples are possible as well.

While the term “upward” is used, it is to be understood that the upwardmovement may encompass any movement having any combination of moving,tilting, rotating, shifting, sliding, or other movement that results ina generally upward movement. Further, in some embodiments “upward” mayrefer to an upward movement in the reference frame of a wearer of thewearable computing device. Other reference frames are possible as well.In embodiments where the wearable computing device is a head-mounteddevice, the upward movement of the wearable computing device may also bean upward movement of a wearer's head such as, for example, the userlooking upward.

The movement data corresponding to the upward movement may take severalforms. For example, the movement data may be (or may be derived from)data received from one or more movement sensors, accelerometers, and/orgyroscopes configured to detect the upward movement, such as the sensor122 described above in connection with FIG. 1A. In some embodiments, themovement data may comprise a binary indication corresponding to theupward movement. In other embodiments, the movement data may comprise anindication corresponding to the upward movement as well as an extent ofthe upward movement. The movement data may take other forms as well.

FIG. 5B shows aspects of an example user-interface after receivingmovement data corresponding to an upward movement. As shown, theuser-interface 500 includes the view region 502 and the menu 504.

As noted above, in response to receiving the movement data correspondingto an upward movement of the wearable computing device, the wearablecomputing device may move one or both of the view region 502 and themenu 504 such that the menu 504 becomes more visible in the view region502.

As shown, the menu 504 is fully visible in the view region 502. In otherembodiments, however, only a portion of the menu 504 may be visible inthe view region 502. In some embodiments, the extent to which the menu504 is visible in the view region 502 may be based at least in part onan extent of the upward movement.

Thus, the view region 502 may be moved in response to receiving datacorresponding to an upward movement. In some embodiments, the viewregion 502 may be moved in an upward scrolling or panning motion. Forinstance, the view region 502 may appear to a wearer of the wearablecomputing device as if mapped onto the inside of a static spherecentered at the wearable computing device, and movement of the viewregion 502 may map onto movement of the real-world environment relativeto the wearable computing device. A speed, acceleration, and/ormagnitude of the upward scrolling may be based at least in part on aspeed, acceleration, and/or magnitude of the upward movement. In otherembodiments, the view region 502 may be moved by, for example, jumpingbetween fields of view. In still other embodiments, the view region 502may be moved only when the upward movement exceeds a threshold speed,acceleration, and/or magnitude. In response to receiving datacorresponding to an upward movement that exceeds such a threshold orthresholds, the view region 502 may pan, scroll, slide, or jump to a newfield of view. The view region 502 may be moved in other manners aswell.

While the foregoing description focused on upward movement, it is to beunderstood that the wearable computing device could be configured toreceive data corresponding to other directional movement (e.g.,downward, leftward, rightward, etc.) as well, and that the view region502 may be moved in response to receiving such data in a manner similarto that described above in connection with upward movement.

As shown, the menu 504 includes a number of content objects 506. In someembodiments, the content objects 506 may be arranged in a ring (orpartial ring) around and above the head of a wearer of the wearablecomputing device. A ring or partial ring in this sense should not beconsidered necessarily circular, but rather may be represented by anytype of arc, ellipse, or combination of arcs. In other embodiments, thecontent objects 506 may be arranged in a dome-shape above the wearer'shead. The ring or dome may be centered above the wearable computingdevice and/or the wearer's head. In other embodiments, the contentobjects 506 may be arranged in other ways as well.

The number of content objects 506 in the menu 504 may be fixed or may bevariable. In embodiments where the number is variable, the contentobjects 506 may vary in size according to the number of content objects506 in the menu 504. In embodiments where the content objects 506 extendcircularly around a wearer's head, like a ring (or partial ring), onlysome of the content objects 506 may be visible at a particular moment.In order to view other content objects 504, a wearer of the wearablecomputing device may interact with the wearable computing device to, forexample, rotate the content objects 506 along a path (e.g., clockwise orcounterclockwise) around the wearer's head. To this end, the wearablecomputing device may be configured to receive data indicating such aninteraction through, for example, a touch pad, such as finger-operabletouch pad 124. Alternatively or additionally, the wearable computingdevice may be configured to receive such data through other inputdevices as well.

Depending on the application of the wearable computing device, thecontent objects 506 may take several forms. For example, the contentobjects 506 may include one or more of people, contacts, groups ofpeople and/or contacts, calendar items, lists, notifications, alarms,reminders, status updates, incoming messages, recorded media, audiorecordings, video recordings, photographs, digital collages,previously-saved states, webpages, and applications, as well as tools,such as a still camera, a video camera, and an audio recorder. Contentobjects 506 may take other forms as well.

In embodiments where the content objects 506 include tools, the toolsmay be located in a particular region of the menu 504, such as thecenter. In some embodiments, the tools may remain in the center of themenu 504, even if the other content objects 506 rotate, as describedabove. Tool content objects may be located in other regions of the menu504 as well.

The particular content objects 506 that are included in menu 504 may befixed or variable. For example, the content objects 506 may bepreselected by a wearer of the wearable computing device. In anotherembodiment, the content objects 506 for each content region may beautomatically assembled by the wearable computing device from one ormore physical or digital contexts including, for example, people,places, and/or objects surrounding the wearable computing device,address books, calendars, social-networking web services orapplications, photo sharing web services or applications, searchhistories, and/or other contexts. Further, some content objects 506 mayfixed, while the content objects 506 may be variable. The contentobjects 506 may be selected in other manners as well.

Similarly, an order or configuration in which the content objects 506are displayed may be fixed or variable. In one embodiment, the contentobjects 506 may be pre-ordered by a wearer of the wearable computingdevice. In another embodiment, the content objects 506 may beautomatically ordered based on, for example, how often each contentobject 506 is used (on the wearable computing device only or in othercontexts as well), how recently each content object 506 was used (on thewearable computing device only or in other contexts as well), anexplicit or implicit importance or priority ranking of the contentobjects 506, and/or other criteria.

In some embodiments, the wearable computing device may be furtherconfigured to receive from the wearer a selection of a content object506 from the menu 504. To this end, the user-interface 500 may include acursor 508, shown in FIG. 5B as a reticle, which may be used to navigateto and select content objects 506 from the menu 504. In someembodiments, the cursor 508 may be controlled by a wearer of thewearable computing device through one or more predetermined movements.Accordingly, the wearable computing device may be further configured toreceive selection data corresponding to the one or more predeterminedmovements.

The selection data may take several forms. For example, the selectiondata may be (or may be derived from) data received from one or moremovement sensors, accelerometers, gyroscopes, and/or detectorsconfigured to detect the one or more predetermined movements. The one ormore movement sensors may be included in the wearable computing device,like the sensor 122, or may be included in a peripheral devicecommunicatively coupled to the wearable computing device. As anotherexample, the selection data may be (or may be derived from) datareceived from a touch pad, such as the finger-operable touch pad 124described above in connection with FIG. 1A, or other input deviceincluded in or coupled to the wearable computing device and configuredto detect one or more predetermined movements. In some embodiments, theselection data may take the form of a binary indication corresponding tothe predetermined movement. In other embodiments, the selection data mayindicate the extent, the direction, the velocity, and/or theacceleration associated with the predetermined movement. The selectiondata may take other forms as well.

The predetermined movements may take several forms. In some embodiments,the predetermined movements may be certain movements or sequence ofmovements of the wearable computing device or peripheral device. In someembodiments, the predetermined movements may include one or morepredetermined movements defined as no or substantially no movement, suchas no or substantially no movement for a predetermined period of time.In embodiments where the wearable computing device is a head-mounteddevice, one or more predetermined movements may involve a predeterminedmovement of the wearer's head (which is assumed to move the wearablecomputing device in a corresponding manner). Alternatively oradditionally, the predetermined movements may involve a predeterminedmovement of a peripheral device communicatively coupled to the wearablecomputing device. The peripheral device may similarly be wearable by awearer of the wearable computing device, such that the movement of theperipheral device may follow a movement of the wearer, such as, forexample, a movement of the wearer's hand. Still alternatively oradditionally, one or more predetermined movements may be, for example, amovement across a finger-operable touch pad or other input device. Otherpredetermined movements are possible as well.

As shown, a wearer of the wearable computing device has navigated thecursor 508 to the content object 506 using one or more predeterminedmovements. In order to select the content object 506, the wearer mayperform an additional predetermined movement, such as holding the cursor508 over the content object 506 for a predetermined period of time. Thewearer may select the content object 506 in other manners as well.

Once a content object 506 is selected, the wearable computing device maycause the content object 506 to be displayed in the view region 502 as aselected content object. FIG. 5C shows aspects of an exampleuser-interface after selection of a selected content object, inaccordance with an embodiment.

As indicated by the dotted arrow, the content object 506 is displayed inthe view region 502 as a selected content object 510. As shown, theselected content object 510 is displayed larger and in more detail inthe view region 502 than in the menu 504. In other embodiments, however,the selected content object 510 could be displayed in the view region502 smaller than or the same size as, and in less detail than or thesame detail as, the menu 504. In some embodiments, additional content(e.g., actions to be applied to, with, or based on the selected contentobject 510, information related to the selected content object 510,and/or modifiable options, preferences, or parameters for the selectedcontent object 510, etc.) may be showed adjacent to or nearby theselected content object 510 in the view region 502.

Once the selected content object 510 is displayed in the view region502, a wearer of the wearable computing device may interact with theselected content object 510. For example, as the selected content object510 is shown as an email inbox, the wearer may wish to read one of theemails in the email inbox. Depending on the selected content object, thewearer may interact with the selected content object in other ways aswell (e.g., the wearer may locate additional information related to theselected content object 510, modify, augment, and/or delete the selectedcontent object 510, etc.). To this end, the wearable computing devicemay be further configured to receive input data corresponding to one ormore predetermined movements indicating interactions with theuser-interface 500. The input data may take any of the forms describedabove in connection with the selection data.

FIG. 5D shows aspects of an example user-interface after receiving inputdata corresponding to a user input, in accordance with an embodiment. Asshown, a wearer of the wearable computing device has navigated thecursor 508 to a particular subject line in the email inbox and selectedthe subject line. As a result, the email 512 is displayed in the viewregion, so that the wearer may read the email 512. The wearer mayinteract with the user-interface 500 in other manners as well, dependingon, for example, the selected content object.

4. Exemplary Methods and Systems for Collision Avoidance

FIG. 6 is a simplified flow chart illustrating a method according to anexemplary embodiment. In particular, a wearable computer having a HMDmay implement method 600 in order to initiate a collision-avoidanceaction in response to a nearby object.

More specifically, method 600 involves the wearable computer displayinga user-interface, which includes a view region and at least one contentregion that is located outside of the view region, on a substantiallytransparent display of an HMD, as shown by block 602. Initially, thestate of the user-interface is such that the view region substantiallyfills the field of view of the HMD, and the at least one content regionis not fully visible in the field of view. In this initial state, thewearable computer displays one or more virtual objects in the viewregion, as shown by block 604. The wearable computer then uses data fromone or more first sensors as a basis for determining a distance betweenthe HMD and a physical object, as shown by block 606. After determiningthe distance between the HMD and the physical object, the wearablecomputer may use the determined distance as basis for initiating acollision-avoidance action that includes de-emphasizing at least one ofthe displayed virtual objects so as to provide a less-obstructed view ofthe physical object through the transparent display of the HMD, as shownby block 608.

An exemplary wearable computer may use proximity and/orrelative-movement data for nearby physical objects to determine when acollision is possible and responsively unclutter the wearable computer'sHMD. For instance, an exemplary wearable computer may use proximitydata, which may indicate the respective distance from the wearablecomputer (or from sensors associated with the wearable computer) tonearby objects. This proximity data may then be used to determine whenan object is too close, and to responsively unclutter the HMD (e.g., bysubstantially removing virtual objects from the wearer's field of view).

In addition to proximity data, some embodiments may analyzerelative-movement data, which may indicate a real-world object'svelocity and/or direction of movement relative to the wearable computer(or relative to sensors associated with the wearable computer). The term“relative movement” in this case may refer to the velocity and/or theacceleration of an object relative to the wearable computer measuringsuch data (e.g., relative to the sensors of the wearable computermeasuring such data). As such, the relative movement of an object mayalso be considered to be the movement of the object relative to thewearer of such a wearable computer. Thus, an object's relative movementmay depend not only on its own motion, but also on the motion of thewearable computer (or sensors of the wearable computer).

As noted, in some embodiments, the wearable computer may use thecombination of proximity data and relative-movement data to determinewhether a risk of collision with a nearby object is such that the HMDshould be uncluttered. For example, the wearable computer may implementa variable threshold distance for collision avoidance, which dependsupon the relative velocity of an object towards the wearable computer.In particular, the wearable computer may adjust the threshold distanceaccording to the relative movement and/or direction of a nearby object.For instance, a wearable computer may generally reduce the thresholddistance at which collision avoidance is initiated as (a) the nearbyobject's relative velocity towards the wearable computer increasesand/or (b) as the nearby object's trajectory indicates a collision ismore likely, and vice versa. An alternative implementation may utilizemovement and distance data to determine a time-until-collision, and thenuse this time-until-collision as a basis for initiating acollision-avoidance action.

As a specific example, a first object that is approaching from tenmeters away with a relative velocity of two meters per second may beassigned a determined distance that is half of the determined distanceassigned to a second object approaching from ten meters away at arelative velocity of one meter per second. Because the first object inthis example may pose a more immediate risk of collision, an exemplarymethod may advantageously initiate a more urgent collision-avoidanceaction. Further, some exemplary systems may be configured to onlyactivate a collision-avoidance action in response to objects that areapproaching the display regardless of determined distance. Otherexamples are also possible.

Some exemplary methods may involve detecting physical hazards which areoutside of the field of view and initiating collision-avoidance actionsin response to such hazards. For example, the HMD may be approaching anobject that is too low along the ground to be seen through the displaywindow (e.g., a curb, a stump, a stair). Although the virtual objects onthe HMD may not be actively obscuring the view of this object, theobject may present a tripping hazard for a wearer of the HMD

a. Determining Distance

The wearable computer may use with various sensors or combinations ofsensors to acquire the data that is used to initiate acollision-avoidance action. For example, some embodiments may utilizedata from video cameras. As a specific example, an HMD may includefront-facing cameras, which may be configured to capture images thatextend beyond the field of view provided in the view region. Then, anintegral or remotely attached computing device may employ imageprocessing techniques to determine that a portion of the captured imagerepresents a physical object and further estimate the distance betweenthe camera and the object. In this case, the camera and wearablecomputer may transmit this estimated distance data to the computingsystem carrying out an object-detection process. Hence the objectdetection process may use the already determined distance as a basis foractivating a collision-avoidance action.

As another example, some embodiments may utilize data from one or moreacoustic or optical sensors when evaluating whether to initiate acollision-avoidance action. For instance, an exemplary embodiment mayincorporate an active acoustic sensing procedure to determine theproximity of a real-world object. Such a procedure may involve emittinga well-defined acoustic pulse and then detecting reflected acousticwaves returning to the system (i.e., echoes). Additionally oralternatively, some embodiments may incorporate an active opticalsensing procedure involving emission of a well-defined light wave,typically outside of the visible spectrum, and detection of reflectedoptical waves. In either case, an exemplary system may use suchtechniques to detect the existence and determine the proximity of nearbyobjects. Further, such techniques may be used to track the movement ofthese physical objects over time (e.g., by periodically repeating suchtechniques to determine movement).

Any of many exemplary sensors may be employed for data collection. Insome cases, the sensors may process the raw data to determine theobject's distance before sending out this determination to the processorresponsible for the collision-avoidance action. In other cases, theprocessor may receive the raw data and process it to determine theobject's distance.

In some embodiments, data received from the one or more distance sensorsmay further be used to determine the position of the physical objectaround the HMD. For example, sensors may be configured to detect theangle that a line between the physical object and the HMD would makewith respect to the ground and with respect to the direction that theHMD is facing. In this way, the position of the physical object could bedetermined in three dimensions (i.e., using the two angles and therelative distance as spherical coordinates). Then, the positiondetermination may be used to track the portion of the HMD that overlaysthe physical object. For example, if the direction of the vector betweenthe HMD and the physical object is slightly left and below the directionof a vector representing the forward direction of the HMD, then theobject may be most obscured by virtual objects in the lower left sectionof the HMD. Determination of the overlaying portion may help enable theHMD to determine the most appropriate collision-avoidance actions to beapplied in response. In the example of an object below and to the leftof center, some exemplary methods may only affect virtual objects withinthe lower left portion of the screen.

b. Determining Relative Movement

As noted, in some embodiments, an object's relative movement may be usedas a basis for adjusting the determined distance of the object that willbe used to initiate a collision-avoidance action. Accordingly, varioustypes of sensors may be employed, alone or in combination, in order todetermine the relative movement of a nearby object.

In some embodiments, a second set of sensors may be used to collect andreport the relative-movement data. In practice, this second set ofsensors may include the first set of sensors, either in part orentirely. For example, an active optical detection system may emit ashort monochromatic pulse and measure the frequency, phase shift, andtime delay of the returning wave. The time delay and phase shift may beused to determine the distance of the reflecting object. Additionally,the frequency and phase change may be used to determine the relativevelocity of the object. In this example, the first and second sensorsmay in fact be the same sensors. However, it also possible that thesensors used to determine relative-movement data may be separate fromthose used to determine proximity data.

c. Initiating Collision-Avoidance Actions

A collision-avoidance action may be activated in response to a varietyof factors. For example, determination that a physical object is withina given distance threshold may be used as a basis for activating thecollision-avoidance action. Also, determination that a physical objectis moving sufficiently towards an HMD at a relative velocity higher thana given velocity threshold may be used as a basis for activating thecollision-avoidance action. As another example, a physical object may bedetermined to be accelerating in the direction of an HMD. Determinationthat a wearable computer is in motion or that the view region of theuser-interface is sufficiently cluttered may also be used as a basis foractivating a collision-avoidance action. Further, any or all of thesefactors may be combined to form bases for activating acollision-avoidance action. For instance, an exemplary embodiment mayactivate a collision-avoidance action in response to receiving data froman accelerometer indicating that the wearable computer is in motionabove a certain speed threshold where the certain speed threshold isdetermined from data indicating that the percentage of the view regionthat is occupied.

In some embodiments, a physical object's determined distance may be usedas a basis for activating a collision-avoidance action. For example, amethod may set a distance threshold for all objects. Then, acollision-avoidance action may be activated in response to determinationthat a physical object is within this distance threshold. In someexemplary systems and methods, multiple distance thresholds may beassociated with multiple collision-avoidance actions.

In some embodiments, distance determination or collision avoidanceactions using distance as a basis for activation may only apply tophysical objects sufficiently overlaid by the HMD. For example, a systemmay determine that a physical object is near to an HMD, but that view ofthe object is not overlaid by the HMD. In this example, acollision-avoidance action that de-emphasizes virtual objects displayedto the HMD would not provide a less obstructed view of the physicalobject. Therefore, an exemplary system may determine not to use theproximity of this physical object as a basis for activating thecollision-avoidance action. Alternatively, some exemplary embodimentsmay determine the proximity of all detected physical objects as a basisfor activating a collision-avoidance action.

Some embodiments may use determination of the relative movement of anobject as a basis for activating a collision-avoidance action. Forinstance, some exemplary embodiments may be configured to activate acollision-avoidance action only in response to objects that aresufficiently approaching the display. In such cases, determined distancemay still be used as a secondary basis for activating acollision-avoidance action, but only after the condition of determinedrelative movement is satisfied. Some embodiments may include setting arelative velocity threshold in which physical objects determined to bemoving sufficiently towards the HMD with a velocity higher than athreshold velocity may be the basis for activating a collision-avoidanceaction. Likewise, if an object is determined to be accelerating towardsthe HMD at greater than a threshold rate of acceleration, an exemplarywearable computer may responsively initiate a collision-avoidanceaction. Other exemplary movement patterns may also be used.

As noted above, in some embodiments, an object's relative movement maybe used as a basis for adjusting the threshold distance of a physicalobject at which a collision-avoidance action is initiated. For instance,again consider that a first object moving toward a sensor with arelative velocity of two meters per second and a second object movingtoward the sensor with a relative velocity of one meter per second mayeach have a determined distance of ten meters from an HMD. However, anexemplary embodiment may assign the first object a threshold distancefor of twelve meters based on the first object's relative movement. Thesecond object may be assigned a threshold distance of eight meters basedon its relative movement. Then, a collision-avoidance action may beactivated in response to detection of the first object at ten meters,but not in response to detecting the second object at the same distance.Note that the specific distances and adjustments in this and otherexamples herein are provided for illustrative purposes, and should notbe considered limiting. It is contemplated that the manner in which thedistance and or movement of a nearby object are used to trigger acollision-avoidance action may vary, depending upon the specificimplementation.

In some embodiments, a determined time-until-collision value may be usedas a basis for initiation of a collision-avoidance action. Using theexample from the section above, the first object would have an initialtime-until-collision value of five seconds and the second object wouldhave an initial time-until-collision value of ten seconds. If the systemdetecting these objects has a threshold time-until-collision of eightseconds, only the first object will initially surpass the threshold andactivate the collision-avoidance action. The time-until-collision may befurther determined or changed based on higher-order movement data (e.g.,acceleration, jerk, etc.) For example, if the second object in the aboveexample, which is initially approaching at one meter per second, is alsoaccelerating towards the HMD at a rate of one meter per second persecond (1 m/ŝ2), then the second object may actually reach the HMD infour seconds. Hence, the HMD may use relative movement data toanticipate this faster approach and assign a shortertime-until-collision for this object.

In some embodiments, the determination that an HMD is in motion may beused as a basis for activation of a collision-avoidance action. Forexample, a system may be configured to activate a collision-avoidanceaction only when the HMD is moving above a threshold speed. Such anembodiment may also use other bases for activation, but only after theHMD surpasses the movement threshold. Some embodiments may be configuredto begin detecting physical objects only after determining that the HMDis in motion.

Some embodiments may use a level of clutter as a further basis foractivating a collision-avoidance action. For example, a determination ofthe level of clutter may include receiving an indication of the amountof unoccupied area in the view region of a user-interface. As anotherexample, the number and size of virtual objects displayed in the viewregion may be used as bases for the level of clutter. An embodiment maythen activate a collision-avoidance action only when a threshold levelof clutter is surpassed.

Predefined user-preferences may also indicate limits on acollision-avoidance action (for example, a time limit,collision-avoidance actions allowed, specified velocity and accelerationthresholds, etc.). As an exemplary user-preference, a system may beconfigured to activate a collision-avoidance action only when the viewregion of a user-interface is sufficiently cluttered. For instance, ifonly a small virtual object is displayed near the edges of the viewregion, the system may determine that no collision-avoidance action needbe taken in response to a detected physical object.

d. De-Emphasizing Virtual Objects

Exemplary collision-avoidance actions may include de-emphasizing one ormore virtual objects displayed in the view region of a user-interface.De-emphasizing virtual objects may include, for example, removing avirtual object that is determined to overlay the physical object,removing all virtual objects from the view region, moving virtualobjects within the view region, moving virtual objects to areaspartially within and partially without the view region, reducing thesize of virtual objects, and making virtual objects at least partiallytransparent. Additionally, exemplary de-emphasizing actions may includeportraying the changes to virtual objects in many ways.

5. Exemplary Collision-Avoidance User-Interfaces

Numerous potential collision-avoidance actions may be applied once anearby physical object is detected. For example, a number of methods maybe employed to de-emphasize one or more virtual objects. Additionally,new virtual objects, audio alarms, and/or physical indications may beused to better attract attention to the detected physical object.Further, exemplary collision-avoidance actions may be combined tomaximize effect.

FIG. 7A is a simplified illustration of a user-interface, according toan exemplary embodiment. In particular, FIG. 7A illustrates a state ofuser-interface 700 in which a collision-avoidance action may be applied.As shown, the user-interface 700 provides virtual objects 706 and 708 ina view region 702. The user-interface 700 also provides content indicia712A, 712B, and 712C in a content region 704 which may be outside of thedisplayed portion of the user-interface 700. Additionally, some of thephysical field of view beyond the display is shown in FIG. 7A, includinga physical object 710 near the display. As shown in FIG. 7A, the view ofthe physical object is substantially obstructed by virtual object 706and virtual object 708.

FIGS. 7B to 7H are simplified illustrations of the user-interface shownin FIG. 7A in different states from that shown in FIG. 7A. Inparticular, each of FIGS. 7B to 7H illustrates user-interface 700 aftercarrying out an exemplary collision-avoidance action.

More specifically, FIG. 7B illustrates the results of an exemplarycollision-avoidance action in which virtual object 706 has been removedfrom the view region 702 of the user-interface 700. Additionally, a newcontent object 714 corresponding to virtual object 706 has been providedin the content region 704. FIG. 7B illustrates that the physical object710 is no longer substantially obscured, because virtual object 708 onlycovers a relatively small portion of the view of physical object 710.Subsequent to the collision-avoidance action activation, a user mayselect content object 714 from the content region and the system mayresponsively return virtual object 706 to the view region 702. In somecases, a content object representing the content of virtual object 706may already occupy the content region while virtual object 706 isdisplayed in the view region 702. In such an instance, this exemplarycollision avoidance action would simply remove virtual object 706 fromthe view region 702, without creating a new content object 714.

FIG. 7C illustrates an exemplary collision-avoidance action, in whichboth virtual object 706 and virtual object 708 are removed from the viewregion 702 of the user-interface 700. As in the above technique, if nocontent object corresponding to virtual object 706 is provided in thecontent region 704, such a content object 714 may be created andprovided in the content region 704. In this procedure, a content object716 corresponding to virtual object 708 may also be provided to thecontent region 704. In addition to allowing a further unobstructed viewof the physical object 710, this technique may more fully draw theattention of a user than the technique in which only a single virtualobject is removed from display. Advantageously, this collision-avoidanceaction may not require the object-detection procedure to determine thelocation of the physical object beyond the view region, as it might inorder to apply the technique depicted in FIG. 5B.

In addition to the embodiments pictured in FIG. 7B and FIG. 7C, contentobjects representing each virtual object may or may not be placed in athe content region 704 for every virtual object removed from the viewregion 702. For example, an exemplary user-interface may include ahistory region, in which content objects representing recently closedcontent, in addition to the content region. In this case, a contentobject representing a removed virtual object may be created in thehistory region in addition to or instead of placing the content objectin the content region 704. Further, removal of a virtual object may beaccompanied by no new content object at all.

FIG. 7D illustrates the result of an exemplary collision-avoidanceaction in which virtual object 706 and virtual object 708 are movedwithin the view region 702 instead of being removed from the viewregion. As shown in FIG. 7D, the physical object 710 may be mostlyvisible as a result of simply moving the virtual objects 706 and 708 offto the sides of the view region 702. In some cases, movement may followa pre-defined course. For example, a display may be configured to alwaysmove objects laterally towards the nearest lateral edge of the screen(as may be the case for the example of FIG. 7D). Alternatively, awearable computer may be configured to specify an appropriate movementpattern based on the particular portion of the display that is currentlycovering the view of the physical object.

FIG. 7E illustrates the result of another exemplary collision-avoidanceaction, in which virtual objects 706 and 708 are moved within the viewregion 702. In contrast to the process illustrated by FIG. 7D, theprocess illustrated by FIG. 7E may move virtual objects 706 and 708 to alocation at least partially outside of the view region 702. In anexemplary system, the wearable computer may be configured to storepositions of virtual objects outside of a default view region inuser-interface 700. As described above, the wearable computer may alsobe configured to change the portion of the user-interface 700 that isprovided to the display. In this way, a system may provide an accessibleuser-interface that is in effect larger than the displayed view region.If, therefore, a virtual object is moved partially outside of the viewregion, the user-interface may provide a process for a user to continueexamining the virtual object. For example, an HMD may be configured sothat if an object is moved away from the center of the display to theleft, and the HMD then is turned to the left, the object will beresponsively brought back towards the center of the view region inaccordance with the movement of the HMD.

As shown in FIG. 7E, the content of the virtual objects may remainintact while the virtual object is moved partially outside of the viewregion, as illustrated by virtual object 706. Alternatively, the contentof the virtual object may be altered, as illustrated by altered virtualobject 708. In this example, the title and a brief preview of thecontent of virtual object 708 are preserved in the view region, to allowfor easy recognition of this object by a user. Many other contentalterations may be applied to such a situation.

FIG. 7F illustrates the result of an exemplary collision-avoidanceaction in which virtual object 706 and virtual object 708 are stilldisplayed in the view region 702 at roughly their original positions,but have been reduced in size. By reducing the size of virtual objects706 and 708, the physical object 710 may be seen and virtual objects 706and 708 may remain in their original positions. Of course, if thisprocess is combined with another de-emphasizing technique, the virtualobjects may be moved from their original positions or otherwise affectedin addition to the shrinking process. As with the de-emphasizingtechnique illustrated in FIG. 7E, the content of each virtual object maybe altered in several ways in response to a reduction in the virtualobject's size. For example, the content of virtual object 706 in FIG. 7Fhas been reduced in size uniformly, as with the shrinking of a singlepicture. As another example, the content of virtual object 708 has beencondensed into the title and a brief excerpt from the content. Anexemplary technique may include procedures for receiving user-inputselecting a minimized virtual object and responsively restoring thevirtual object to its original size. In this way, virtual object 708 maybehave in similar manner to a content object being shown in the viewregion. Many other content reduction processes may be employed. Forinstance, content alteration procedures may be selected based on thecontent of a particular virtual object, the amount of shrinkingexpected, and/or user-preferences.

FIG. 7G illustrates the result of an exemplary collision-avoidanceaction in which virtual objects 706 and 708 are made at least partiallytransparent so that physical object 710 may be more easily visiblethrough the transparent display of the HMD. An exemplary embodiment mayapply transparency only to objects that are determined to besufficiently obscuring the physical object or, alternatively, to allvirtual objects within the view region (as depicted in FIG. 7G).Additionally, some embodiments may be configured to make only a portionof each virtual object transparent. For example, if a single virtualobject covers the entire screen, an exemplary technique may cause awindow of the virtual object to become transparent directly in front ofa detected physical object while leaving the remainder of the virtualobject substantially opaque.

In addition to moving and/or de-emphasizing virtual objects, anexemplary collision-avoidance action may involve adding one or morevirtual objects to highlight the detected physical object. FIG. 7Hillustrates the results of one such a collision-avoidance action. Morespecifically, in addition to moving virtual object 706 and virtualobject 708, the user-interface has presented a new virtual object 718 todraw attention to the approaching physical object 710. This example newvirtual object 718 is shown as a caution symbol and a border around thedetected object. However, a highlighting object may take many forms.Additionally, this technique need not be combined with other proceduresif the physical object is already partially visible. Further, if aphysical object is detected outside of the field of view beyond thedisplay, the new virtual object may include a directional symbol or someother indication of where the potential obstacle is with respect to thedisplay.

An exemplary collision-avoidance action may further specify theparticular new virtual object to be presented based on characteristicsof the detected physical object. For example, new virtual object 718from FIG. 7H includes a border around the detected object. If adifferent physical object were detected, this border would need to bereshaped to encompass this different object. If the detected physicalobject were outside of the field of view covered by the display window,a new virtual object indicating the direction of the detected object maybe presented as part of a collision-avoidance action. Further, if one ormore detected physical objects may be most easily avoided by the wearerof the HMD moving in a particular way (e.g., moving left, ducking down,stepping up), a new virtual object, such as an arrow pointing in themovement direction and/or text instructing the wearer the way to move,may be presented.

Exemplary collision-avoidance actions may also include nonvisual alarmsto alert a wearer. For example, a wearable display may include speakers,which may present an audio alarm to complement a visual cue.Advantageously, such an alarm could alert a user without needing tochange the visual display. As with presenting new virtual objects, anonvisual alarm may be specific to the situation. For example, if anobject is determined to be best avoided by a wearer moving left, anaudio alert may include a voice saying, “Move left”. Additionally,non-visual cues could include physical cues such as vibration of thesystem or physical movement of display windows.

Exemplary embodiments may be further configured to combine any or allcollision-avoidance actions to most effectively handle a collisionavoidance action. For example, virtual objects may be both moved andturned partially transparent so that a physical object may be whollyvisible and the surrounding field of view be made more visible than ifthe objects were simple moved. As another example, a collision-avoidanceaction may de-emphasize a virtual object in successive steps if initialattempts have not produced a desired affect (e.g., a wearer continues toapproach a detected object in a potentially hazardous way despite aninitial collision-avoidance action.) For instance, an exemplary processmay include receiving and indication that an object is within a firstthreshold distance and responsively moving one virtual object to providea view of the object. The process may further include receiving asubsequent indication that the detected object is within a second,closer distance threshold and responsively moving all virtual objects toprovide a less obstructed view of the approaching object. Upon receivingadditional indications that the object continues to pose the risk ofcollision, the exemplary process may include applying transparency toone or all of the virtual objects, reducing the virtual objects in size,removing the virtual objects from the viewing region entirely, andfinally presenting a new virtual object to highlight the physicalobject. It will become obvious to persons of skill in the art to combinecollision-avoidance action in many other advantageous ways.

An exemplary display may be configured to portray the moving or removingof virtual objects in various ways. For example, virtual objects thatare moved from position to position within the viewing region may beshown successively at intermediate positions, to portray a patternsimilar to the movement of a physical object. Such a procedure may alsobe applied when an object is removed from the view region and acorresponding content object is created in the content region. In thiscase, the object may graphically show successive progress toward thecontent region until the virtual object is completely removed from theview region. Another exemplary process for portraying the removal of avirtual object may include gradually making the object more transparentuntil it vanishes from display. As another example, virtual objects maybe portrayed as reducing in size before being removed from the viewregion. Further, virtual objects may be portrayed as increasing in sizebefore being removed from the view region to give the impression thatthe virtual objects are moving through the display to a point behind awearer.

Another exemplary portrayal technique may help to allow a wearer toutilize their own instinctive collision-avoidance maneuvers by movingthe virtual objects in such a way as to indicate a quickly approachingobstacle. For example, if the virtual objects sweep rapidly towards theedges of the screen and increase in size, a wearer may perceive thatphysical objects in their surroundings are closer or approaching morequickly than if virtual objects simply disappeared. Hence, such amovement pattern may portray an imminent danger to the wearer, allowingthe wearer to make a more immediate adjustment to avoid a physicalobject. In this example, a computer may select the edge of the screen towhich the virtual objects move based on the relative position of thephysical object. For instance, if a physical object is detected in frontof the HMD but slightly left of the field of view's center, the virtualobjects may be expanded and moved off to the left of the screen. Asanother example, if the relative motion of a physical object is directedto a point only slightly left of the HMD, the objects may likewise sweeptoward the left edge of the screen. In some embodiments, virtual objectsmay expand in size while becoming increasingly transparent until beingremoved entirely from the screen. The virtual objects in this case wouldnot need to move toward a screen edge, but could simply be expanded inplace, giving the impression that the virtual objects are passingthrough the HMD to a point behind a wearer.

Some exemplary procedures may include steps for de-activating thecollision-avoidance action in response to receiving indication that thephysical object is no longer a hazard or as a result of user-input.Exemplary collision-avoidance actions may therefore include proceduresto move the virtual objects, which occupied the view region beforeactivation of the collision-avoidance action, back to their originallocations in the view region. Further, such procedures may also includere-establishing the size and opacity of the virtual objects. Forexample, an exemplary de-activation procedure may include storing theoriginal position and content of the virtual objects in the view regionbefore a collision-avoidance action is activated. Then, in response tothe collision-avoidance action being de-activated, the procedure maymove the original content of virtual objects back to their originalpositions in the view region.

Subsequent to initiating the collision-avoidance action, an exemplaryembodiment may also include techniques for de-activating thecollision-avoidance action in response to receiving second data from thesensors. For example, a system may activate a collision-avoidance actionin response to determining that a stationary physical object has arelative movement pointed directly at the wearer of an HMD because thewearer is walking towards the stationary object. The wearer, alerted bythe collision-avoidance action, may adjust course to avoid thestationary object. Then, the system may determine that the object nolonger has a relative movement directed at the HMD and responsivelyde-activate the collision-avoidance action. As another example, thewearer of an HMD may be standing still when a cyclist suddenly turns acorner within a threshold distance of the HMD. An exemplary system maydetect the nearby cyclist and activate a collision-avoidance action toalert the wearer. Subsequently, the cyclist may move beyond thethreshold distance from the HMD and responsively the system mayde-activate the collision-avoidance action. Many other exemplaryde-activation procedures may be utilized.

The de-activation process may also result from input other than distanceand relative movement data received from the sensors. For example, auser-input may be received from one or more input devices indicatingthat the collision-avoidance action should be de-activated. As aspecific example, a collision-avoidance action may include moving avirtual object off to the left side of the view region or even into aportion of the user-interface beyond the left boundary of the viewregion. A wearer of the display may wish to override thecollision-avoidance action and retrieve the original view of the movedvirtual object. An exemplary system may be configured to receiveindication from motion sensors that the wearer has quickly turned thedisplay to the left and responsively return the virtual object to itsoriginal position. Many other user-input techniques, such asvoice-command, gaze-tracked eye movement, actuation of a switch, orinteraction with a touch pad, may be employed to indicate the need forde-activation of a collision-avoidance action.

While various aspects and embodiments have been disclosed herein, otheraspects and embodiments will be apparent to those skilled in the art.The various aspects and embodiments disclosed herein are for purposes ofillustration and are not intended to be limiting, with the true scopeand spirit being indicated by the following claims.

We claim:
 1. A computer-implemented method comprising: causing ahead-mountable device (HMD) to display a field of view to auser-interface mapped to a sphere, wherein the field of view is moveablewith respect to the user-interface, and wherein the user-interfacecomprises: (i) a view region, and (ii) a set of selectable contentobjects arranged in at least a partial ring; associating aforward-looking head position with a forward-looking location of thefield of view on the sphere, wherein a default arrangement of thespherical user-interface both: (i) locates the view region within theforward-looking field of view, and (ii) locates the set of selectablecontent objects substantially outside the forward-looking field of view,wherein the forward-looking field of view is in a direction that isgenerally parallel to the ground; displaying one or more virtual objectsin the view region; using data from one or more first sensors as a basisfor determining a distance between the HMD and a physical object; andusing the determined distance between the HMD and the physical object asbasis for initiating a collision-avoidance action, wherein thecollision-avoidance action de-emphasizes one or more of the displayedvirtual objects to provide a less-obstructed view of the physicalobject.
 2. The method of claim 1, wherein using the determined distanceas basis for initiating the collision-avoidance action comprises:determining that the physical object is less than a threshold distancefrom the HMD; and responsively initiating the collision-avoidanceaction.
 3. The method of claim 1, further comprising: using data fromone or more second sensors to determine a relative movement of thephysical object with respect to the HMD; and using the relative movementas a further basis for initiating the collision-avoidance action.
 4. Themethod of claim 3, wherein using the relative movement as a furtherbasis for initiating the collision-avoidance action comprises: using therelative movement as a basis for determining the threshold distancebetween the physical object and the HMD; and determining that thephysical object is less than the threshold distance from the HMD andresponsively initiating the collision-avoidance action.
 5. The method ofclaim 3, wherein using the relative movement as a further basis forinitiating the collision-avoidance action comprises: using (a) thedistance between the physical object and the HMD and (b) the relativemovement to determine a time period until collision; and determiningthat the time period until collision is less than a threshold timeperiod and responsively initiating the collision-avoidance action. 6.The method of claim 3, further comprising, subsequent to initiating thecollision-avoidance action: using data from the one or more secondsensors as a basis for determining a subsequent relative movement of thephysical object with respect to the HMD; determining that a direction ofthe subsequent relative movement of the physical object is directedsufficiently away from the HMD such that collision with the physicalobject is unlikely; and in response to determining that the subsequentrelative movement of the physical object is directed sufficiently awayfrom the HMD, re-displaying in the view region the at least one virtualobject that was de-emphasized in the collision-avoidance action.
 7. Themethod of claim 1, further comprising: using data from one or more thirdsensors to determine that the HMD is in motion; and using thedetermination that the HMD is in motion as a further basis forinitiating a collision-avoidance action.
 8. The method of claim 1,further comprising: determining that the view region is sufficientlycluttered; and using the determination that the view region issufficiently cluttered as a further basis for initiating thecollision-avoidance action.
 9. The method of claim 1, whereinde-emphasizing at least one of the virtual objects comprises removing atleast one of the virtual objects from the view region.
 10. The method ofclaim 1, wherein the moving the at least one of said virtual objectscomprises efficiently portraying a potential hazard by moving the atleast one of said virtual objects quickly from a center point of theview region toward outer edges of the view region.
 11. The method ofclaim 10, further comprising: determining a portion of the view regionthat overlays the physical object; wherein the at least one of saidvirtual objects is moved quickly away from the portion of the viewregion that overlays the physical object.
 12. The method of claim 1,wherein the physical object is outside of the field of view provided inthe see-through display, and wherein the collision-avoidance actionfurther comprises displaying a visual indication of a direction of thephysical object.
 13. The method of claim 1, wherein thecollision-avoidance action further comprises presenting an audio alert.14. The method of claim 1, wherein the collision-avoidance actionfurther comprises displaying a visual indication that highlights thephysical object.
 15. A computing system comprising: at least oneprocessor; one or more first sensors operable to generate dataindicative of distances to physical objects in an environment of ahead-mountable device (HMD); data storage comprising logic executable bythe at least one processor to: cause a display of the HMD to display afield of view to a user-interface mapped to a sphere, wherein the fieldof view is moveable with respect to the user-interface, and wherein theuser-interface comprises: (i) a view region, and (ii) a set ofselectable content objects arranged in at least a partial ring;associate a forward-looking head position with a forward-lookinglocation of the field of view on the sphere, wherein a defaultarrangement of the spherical user-interface both: (i) locates the viewregion within the forward-looking field of view, and (ii) locates theset of selectable content objects substantially outside theforward-looking field of view, wherein the forward-looking field of viewis in a direction that is generally parallel to the ground; display oneor more virtual objects in the view region; use data from one or morefirst sensors as a basis for determining a distance between the HMD anda physical object; and use the determined distance between the HMD andthe physical object as basis to initiate a collision-avoidance action,wherein the collision-avoidance action de-emphasizes one or more of thedisplayed virtual objects to provide a less-obstructed view of thephysical object.
 16. The computing system of claim 15, wherein theuser-input comprises detection by sensors that the head-mounted displayhas moved in specified movement pattern.
 17. A non-transitorycomputer-readable medium having stored therein instructions executableby a computing device to cause the computing device to perform functionscomprising: causing a head-mountable device (HMD) to display a field ofview to a user-interface mapped to a sphere, wherein the field of viewis moveable with respect to the user-interface, and wherein theuser-interface comprises: (i) a view region, and (ii) a set ofselectable content objects arranged in at least a partial ring;associating a forward-looking head position with a forward-lookinglocation of the field of view on the sphere, wherein a defaultarrangement of the spherical user-interface both: (i) locates the viewregion within the forward-looking field of view, and (ii) locates theset of selectable content objects substantially outside theforward-looking field of view, wherein the forward-looking field of viewis in a direction that is generally parallel to the ground; displayingone or more virtual objects in the view region; using data from one ormore first sensors as a basis for determining a distance between the HMDand a physical object; and using the determined distance between the HMDand the physical object as basis for initiating a collision-avoidanceaction, wherein the collision-avoidance action de-emphasizes one or moreof the displayed virtual objects to provide a less-obstructed view ofthe physical object.
 18. The non-transitory computer-readable medium ofclaim 17, wherein the functions further comprise: using data from one ormore second sensors to determine a relative movement of the physicalobject with respect to the HMD; and using the relative movement as afurther basis for initiating the collision-avoidance action.
 19. Thenon-transitory computer-readable medium of claim 18, wherein thefunctions further comprise subsequent to initiating thecollision-avoidance action: using data from the one or more secondsensors as a basis for determining a subsequent relative movement of thephysical object with respect to the HMD; determining that a direction ofthe subsequent relative movement of the physical object is directedsufficiently away from the HMD such that collision with the physicalobject is unlikely; and in response to determining that the subsequentrelative movement of the physical object is directed sufficiently awayfrom the HMD, re-displaying in the view region the at least one virtualobject that was de-emphasized in the collision-avoidance action.
 20. Thenon-transitory computer-readable medium of claim 17, wherein the movingthe at least one of said virtual objects comprises efficientlyportraying a potential hazard by moving the at least one of said virtualobjects quickly from a center point of the view region toward outeredges of the view region.